Processing Hansard Documents with Portage

The House of Commons translation task involves handling documents that are transcriptions of parliamentary sessions (Hansard) or of the proceedings of various parliamentary committees. The documents are in XML, and generally contain parts where the source language is English, and parts where it is French. The desired output is a set of similarly-formatted XML documents containing both the original passages and their translations. This task poses several problems for automatic processing with Portage: using specialized models for different sub-genres (eg Finance Committee proceedings versus Hansard), running translation in diffferent directions for different parts of a document, and preserving XML structure across translation. The last of these problems is particularly hard when XMLmarkup occurs within a sentence, because it may be difficult or impossible to identify a corresponding marked-up segment in Portage’s output. This document describes the processing strategy that was used in the Translation Bureau trials of September 2010, and refined shortly afterwards. This includes the translation process only; it does not deal with model training, offline adaptation, or exploiting document structure to improve translation, for instance. The processing strategy used for this task can be divided into parts that are Hansard specific, and parts that could be re-used in other settings. The rest of this document reflects this division: section 2 describes the overall strategy and how the two parts fit together; section 3 describes Hansard-specific steps, and section 4 describes general processing steps.